feat: Add BigQuery Agent Analytics Dashboard #4281
Conversation
Add a comprehensive monitoring dashboard for ADK agent behavior analytics stored in BigQuery. The dashboard provides real-time insights into: - Agent sessions and invocations - LLM performance metrics (latency, tokens, TTFT) - Tool usage and success rates - Error tracking and analysis - Latency analysis with percentiles (P50, P90, P95, P99) Components: - FastAPI backend with BigQuery queries - Embedded HTML/JS frontend with Chart.js visualizations - Data simulation script for testing - Docker and Cloud Run deployment support Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
|
Response from ADK Triaging Agent Hello @caohy1988, thank you for creating this PR! Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). You can find more information at https://cla.developers.google.com/. In addition, this PR is a new feature, could you please associate a GitHub issue with this PR? If there is no existing issue, could you please create one? This information will help reviewers to review your PR more efficiently. Thanks! |
Summary of ChangesHello @caohy1988, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request adds a new BigQuery Agent Analytics Dashboard, providing a real-time monitoring solution for ADK agent behavior. It includes a FastAPI backend, an HTML/JS frontend with visualizations, and supports Docker and Cloud Run deployment. The dashboard offers insights into agent performance, tool usage, error tracking, and session analysis, enabling better observability and debugging. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This is a great addition! A comprehensive analytics dashboard is a very valuable tool for monitoring and understanding agent behavior. The FastAPI backend is well-structured with clear API endpoints, and the inclusion of a data simulation script and Docker support makes it easy to get started.
My review focuses on a few key areas to improve maintainability and performance:
- Separating the frontend code (HTML/JS/CSS) from the Python backend to make it easier to manage.
- Optimizing the BigQuery client instantiation to improve API performance.
- Aligning a default configuration value to prevent potential data discrepancies between the agent and the dashboard.
Overall, this is a fantastic feature. The changes I've suggested should help make it even more robust and maintainable.
| def get_dashboard_html() -> str: | ||
| """Return the dashboard HTML with embedded JavaScript.""" |
There was a problem hiding this comment.
The get_dashboard_html function embeds a very large block of HTML, CSS, and JavaScript as a multiline string. This makes the frontend code extremely difficult to read, maintain, and debug. It also prevents developers from using standard frontend development tools like linters, formatters, and syntax highlighting.
For better maintainability, I strongly recommend separating the frontend assets from the Python code:
- Create a
templatesdirectory and move the HTML into a file likedashboard.html. Use a templating engine like Jinja2 to serve it. - Create a
staticdirectory for your CSS and JavaScript files. - Use FastAPI's
StaticFilesto serve thestaticdirectory (it's already imported but unused).
This approach aligns with standard web development practices and will make the dashboard much easier to work on in the future.
|
|
||
| PROJECT_ID = os.getenv("BQ_AGENT_ANALYTICS_PROJECT") | ||
| DATASET_ID = os.getenv("BQ_AGENT_ANALYTICS_DATASET") | ||
| TABLE_ID = os.getenv("BQ_AGENT_ANALYTICS_TABLE", "agent_events") |
There was a problem hiding this comment.
The default table ID is set to "agent_events", which is inconsistent with the dashboard and simulation code where the default is "agent_events_v2". To ensure data is logged to the correct table by default when the BQ_AGENT_ANALYTICS_TABLE environment variable is not set, this should be updated.
| TABLE_ID = os.getenv("BQ_AGENT_ANALYTICS_TABLE", "agent_events") | |
| TABLE_ID = os.getenv("BQ_AGENT_ANALYTICS_TABLE", "agent_events_v2") |
| def get_bq_client() -> bigquery.Client: | ||
| """Get or create BigQuery client.""" | ||
| if not PROJECT_ID: | ||
| raise HTTPException( | ||
| status_code=500, | ||
| detail="BQ_AGENT_ANALYTICS_PROJECT environment variable not set", | ||
| ) | ||
| return bigquery.Client(project=PROJECT_ID) |
There was a problem hiding this comment.
A new bigquery.Client is instantiated on every API call within get_bq_client(). This is inefficient as it can add overhead for authentication and resource initialization on each request. It's better to create a single client instance when the application starts and reuse it across all requests.
A simple way to fix this is to cache the result of this function. You can use @lru_cache from the functools module. You'll need to add from functools import lru_cache at the top of the file.
from functools import lru_cache
@lru_cache
def get_bq_client() -> bigquery.Client:
# ... function bodyAlternatively, you could manage the client's lifecycle using FastAPI's lifespan events.
2 participants
Summary
Live Demo
🚀 Deployed Dashboard: https://agent-analytics-dashboard-201486563047.us-central1.run.app
Features
Screenshots
The dashboard includes 6 tabs:
Files Added
contributing/samples/bigquery_agent_analytics_demo/dashboard/app.py- FastAPI dashboard applicationcontributing/samples/bigquery_agent_analytics_demo/dashboard/simulate_agent_data.py- Data simulation scriptcontributing/samples/bigquery_agent_analytics_demo/dashboard/Dockerfile- Container buildcontributing/samples/bigquery_agent_analytics_demo/dashboard/docker-compose.yml- Docker orchestrationcontributing/samples/bigquery_agent_analytics_demo/dashboard/requirements.txt- Dependenciescontributing/samples/bigquery_agent_analytics_demo/dashboard/README.md- Documentationcontributing/samples/bigquery_agent_analytics_demo/agent.py- Demo agent with BigQuery pluginTest plan
uvicorn app:app --port 8080python simulate_agent_data.py --num-sessions 10🤖 Generated with Claude Code